Построение воронки продаж в phyton

Воронка продаж — маркетинговая модель, описывающая предполагаемое «путешествие» будущего покупателя от первого знакомства с предложением или товаром до реальной покупки. Воронка - базовый инструмент в арсенале продуктового аналитика, т.к. помогает:

• Контролировать процесс продаж
• Анализировать эффективность работы каждого менеджера
• Анализировать эффективность работы различных каналов привлечения
• Подсчитывать конверсию каждого этапа
• Прогнозировать и строить планы продаж на основе средних значений

Простой и быстрый способ построить воронку на phyton - использовать пакет retentioneering.

Что такое retentioneering

Retentioneering - это библиотека Python для помощи продуктовым и маркетинговым аналитикам,которая упрощает обработку и анализ потоков кликов, потоков событий, траекторий и журналов событий. Вы можете сегментировать пользователей, клиентов (агентов), строить конвейеры машинного обучения для прогнозирования категории агента или вероятности целевого события на основе исторических данных.

Функции

Воронки, которые формируются с использованием retentioneering, можно использовать, чтобы сделать быстрое сравнение конверсии в нескольких группах пользователей, например:

• пользователи, пришедшие из разных каналов
• пользователи из тестовых и контрольных групп в A/B-тесте
• пользователи из разных поведенческих сегментов
• пользователи, сегментированные по региональному признаку
• пользователи, привлекаемые различными менеджерами и т.д.

Документация

Документацию о библитеке вы можете изучить тут: https://github.com/retentioneering/retentioneering-tools

Установка

Для установки пакета в ноутбуке Jupiter выполните код:

!pip install retentioneering

Requirement already satisfied: retentioneering in c:\users\вика\appdata\roaming\python\python38\site-packages (2.0.3.2)
Requirement already satisfied: networkx>=2.6.2 in c:\users\вика\appdata\roaming\python\python38\site-packages (from retentioneering) (2.6.3)
Requirement already satisfied: seaborn>=0.11.0 in c:\users\вика\appdata\roaming\python\python38\site-packages (from retentioneering) (0.11.2)
Requirement already satisfied: decorator>=4.3 in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (5.0.6)
Requirement already satisfied: plotly in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (5.3.1)
Requirement already satisfied: numpy<1.20,>=1.19.5 in c:\users\вика\appdata\roaming\python\python38\site-packages (from retentioneering) (1.19.5)
Requirement already satisfied: umap-learn in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (0.5.1)
Requirement already satisfied: altair in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (4.1.0)
Requirement already satisfied: pandas<=1.3.0,>=1.1.5 in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (1.2.4)
Requirement already satisfied: tqdm in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (4.59.0)
Requirement already satisfied: matplotlib in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (3.3.4)
Requirement already satisfied: scikit-learn>=0.23.2 in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (0.24.1)
Requirement already satisfied: statsmodels>=0.12.0 in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (0.12.2)
Requirement already satisfied: numba==0.53 in c:\users\вика\appdata\roaming\python\python38\site-packages (from retentioneering) (0.53.0)
Requirement already satisfied: pymongo in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (3.12.0)
Requirement already satisfied: scipy in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (1.6.2)
Requirement already satisfied: vega in c:\programdata\anaconda3\lib\site-packages (from retentioneering) (3.5.0)
Requirement already satisfied: llvmlite<0.37,>=0.36.0rc1 in c:\programdata\anaconda3\lib\site-packages (from numba==0.53->retentioneering) (0.36.0)
Requirement already satisfied: setuptools in c:\programdata\anaconda3\lib\site-packages (from numba==0.53->retentioneering) (52.0.0.post20210125)
Requirement already satisfied: cycler>=0.10 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->retentioneering) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->retentioneering) (1.3.1)
Requirement already satisfied: pillow>=6.2.0 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->retentioneering) (8.2.0)
Requirement already satisfied: python-dateutil>=2.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->retentioneering) (2.8.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->retentioneering) (2.4.7)
Requirement already satisfied: six in c:\programdata\anaconda3\lib\site-packages (from cycler>=0.10->matplotlib->retentioneering) (1.15.0)
Requirement already satisfied: pytz>=2017.3 in c:\programdata\anaconda3\lib\site-packages (from pandas<=1.3.0,>=1.1.5->retentioneering) (2021.1)
Requirement already satisfied: joblib>=0.11 in c:\programdata\anaconda3\lib\site-packages (from scikit-learn>=0.23.2->retentioneering) (1.0.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\programdata\anaconda3\lib\site-packages (from scikit-learn>=0.23.2->retentioneering) (2.1.0)
Requirement already satisfied: patsy>=0.5 in c:\programdata\anaconda3\lib\site-packages (from statsmodels>=0.12.0->retentioneering) (0.5.1)
Requirement already satisfied: toolz in c:\programdata\anaconda3\lib\site-packages (from altair->retentioneering) (0.11.1)
Requirement already satisfied: jsonschema in c:\programdata\anaconda3\lib\site-packages (from altair->retentioneering) (3.2.0)
Requirement already satisfied: entrypoints in c:\programdata\anaconda3\lib\site-packages (from altair->retentioneering) (0.3)
Requirement already satisfied: jinja2 in c:\programdata\anaconda3\lib\site-packages (from altair->retentioneering) (2.11.3)
Requirement already satisfied: MarkupSafe>=0.23 in c:\programdata\anaconda3\lib\site-packages (from jinja2->altair->retentioneering) (1.1.1)
Requirement already satisfied: attrs>=17.4.0 in c:\programdata\anaconda3\lib\site-packages (from jsonschema->altair->retentioneering) (20.3.0)
Requirement already satisfied: pyrsistent>=0.14.0 in c:\programdata\anaconda3\lib\site-packages (from jsonschema->altair->retentioneering) (0.17.3)
Requirement already satisfied: tenacity>=6.2.0 in c:\programdata\anaconda3\lib\site-packages (from plotly->retentioneering) (8.0.1)
Requirement already satisfied: pynndescent>=0.5 in c:\programdata\anaconda3\lib\site-packages (from umap-learn->retentioneering) (0.5.4)
Requirement already satisfied: jupyter<2.0.0,>=1.0.0 in c:\programdata\anaconda3\lib\site-packages (from vega->retentioneering) (1.0.0)
Requirement already satisfied: ipywidgets in c:\programdata\anaconda3\lib\site-packages (from jupyter<2.0.0,>=1.0.0->vega->retentioneering) (7.6.3)
Requirement already satisfied: nbconvert in c:\programdata\anaconda3\lib\site-packages (from jupyter<2.0.0,>=1.0.0->vega->retentioneering) (6.0.7)
Requirement already satisfied: qtconsole in c:\programdata\anaconda3\lib\site-packages (from jupyter<2.0.0,>=1.0.0->vega->retentioneering) (5.0.3)
Requirement already satisfied: ipykernel in c:\programdata\anaconda3\lib\site-packages (from jupyter<2.0.0,>=1.0.0->vega->retentioneering) (5.3.4)
Requirement already satisfied: notebook in c:\programdata\anaconda3\lib\site-packages (from jupyter<2.0.0,>=1.0.0->vega->retentioneering) (6.3.0)
Requirement already satisfied: jupyter-console in c:\programdata\anaconda3\lib\site-packages (from jupyter<2.0.0,>=1.0.0->vega->retentioneering) (6.4.0)
Requirement already satisfied: tornado>=4.2 in c:\programdata\anaconda3\lib\site-packages (from ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (6.1)
Requirement already satisfied: ipython>=5.0.0 in c:\programdata\anaconda3\lib\site-packages (from ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (7.22.0)
Requirement already satisfied: jupyter-client in c:\programdata\anaconda3\lib\site-packages (from ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (6.1.12)
Requirement already satisfied: traitlets>=4.1.0 in c:\programdata\anaconda3\lib\site-packages (from ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (5.0.5)
Requirement already satisfied: pickleshare in c:\programdata\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.7.5)
Requirement already satisfied: backcall in c:\programdata\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.2.0)
Requirement already satisfied: colorama in c:\programdata\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.4.4)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in c:\programdata\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (3.0.17)
Requirement already satisfied: jedi>=0.16 in c:\programdata\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.17.2)
Requirement already satisfied: pygments in c:\programdata\anaconda3\lib\site-packages (from ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (2.8.1)
Requirement already satisfied: parso<0.8.0,>=0.7.0 in c:\programdata\anaconda3\lib\site-packages (from jedi>=0.16->ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.7.0)
Requirement already satisfied: wcwidth in c:\programdata\anaconda3\lib\site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=5.0.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.2.5)
Requirement already satisfied: ipython-genutils in c:\programdata\anaconda3\lib\site-packages (from traitlets>=4.1.0->ipykernel->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.2.0)
Requirement already satisfied: jupyterlab-widgets>=1.0.0 in c:\programdata\anaconda3\lib\site-packages (from ipywidgets->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.0.0)
Requirement already satisfied: nbformat>=4.2.0 in c:\programdata\anaconda3\lib\site-packages (from ipywidgets->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (5.1.3)
Requirement already satisfied: widgetsnbextension~=3.5.0 in c:\programdata\anaconda3\lib\site-packages (from ipywidgets->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (3.5.1)
Requirement already satisfied: jupyter-core in c:\programdata\anaconda3\lib\site-packages (from nbformat>=4.2.0->ipywidgets->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (4.7.1)
Requirement already satisfied: terminado>=0.8.3 in c:\programdata\anaconda3\lib\site-packages (from notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.9.4)
Requirement already satisfied: pyzmq>=17 in c:\programdata\anaconda3\lib\site-packages (from notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (20.0.0)
Requirement already satisfied: argon2-cffi in c:\programdata\anaconda3\lib\site-packages (from notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (20.1.0)
Requirement already satisfied: prometheus-client in c:\programdata\anaconda3\lib\site-packages (from notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.10.1)
Requirement already satisfied: Send2Trash>=1.5.0 in c:\programdata\anaconda3\lib\site-packages (from notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.5.0)
Requirement already satisfied: pywin32>=1.0 in c:\programdata\anaconda3\lib\site-packages (from jupyter-core->nbformat>=4.2.0->ipywidgets->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (227)
Requirement already satisfied: pywinpty>=0.5 in c:\programdata\anaconda3\lib\site-packages (from terminado>=0.8.3->notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.5.7)
Requirement already satisfied: cffi>=1.0.0 in c:\programdata\anaconda3\lib\site-packages (from argon2-cffi->notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.14.5)
Requirement already satisfied: pycparser in c:\programdata\anaconda3\lib\site-packages (from cffi>=1.0.0->argon2-cffi->notebook->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (2.20)
Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.5.3)
Requirement already satisfied: testpath in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.4.4)
Requirement already satisfied: mistune<2,>=0.8.1 in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.8.4)
Requirement already satisfied: jupyterlab-pygments in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.1.2)
Requirement already satisfied: bleach in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (3.3.0)
Requirement already satisfied: pandocfilters>=1.4.1 in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.4.3)
Requirement already satisfied: defusedxml in c:\programdata\anaconda3\lib\site-packages (from nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.7.1)
Requirement already satisfied: async-generator in c:\programdata\anaconda3\lib\site-packages (from nbclient<0.6.0,>=0.5.0->nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.10)
Requirement already satisfied: nest-asyncio in c:\programdata\anaconda3\lib\site-packages (from nbclient<0.6.0,>=0.5.0->nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.5.1)
Requirement already satisfied: webencodings in c:\programdata\anaconda3\lib\site-packages (from bleach->nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (0.5.1)
Requirement already satisfied: packaging in c:\programdata\anaconda3\lib\site-packages (from bleach->nbconvert->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (20.9)
Requirement already satisfied: qtpy in c:\programdata\anaconda3\lib\site-packages (from qtconsole->jupyter<2.0.0,>=1.0.0->vega->retentioneering) (1.9.0)

Пример построения воронки

Первые шаги

Установите другие библиотеки, которые потребуются для работы:

import numpy as np
import pandas as pd
import io
import retentioneering
from retentioneering import init_config
import datetime
import plotly.express as px
import plotly.graph_objects as go

Загрузите образец данных, которые можно обработать:

data = pd.read_csv('test_dataset.csv')

Обновите конфигурацию, чтобы определить наименования данных:

retentioneering.config.update({
    'event_col':'event',
    'event_time_col':'timestamp',
    'user_col': 'client_id'
})

Построим матрицу потоков и шагов пользователей

Матрица потоков и шагов - это инструмент, который позволяет быстро получить общее представление о поведении пользователей еще до того, как будет построены воронки.

Посмотрим на примере.

desc_table = data.rete.step_matrix(max_steps=12,
                                  targets=['request_call', 'ENDED'],
                                  accumulated='only')

Что видим

По первому столбцу сразу можно сказать, что пользователи в анализируемой когорте начинают свои сеансы с главной страницы (55%) и из каталога продуктов (36%). На шаге 2 51% пользователей уже завершили свои сеансы и не имеют других событий (строка ENDED на шаге 2 равна 0,51). Мы видим, что 93% пользователей завершают свои сеансы с 6 или менее событиями (строка ENDED на шаге 6 равна 0,93).

Построим открытую воронку

Для примера сделаем сравнение воронок пользователей, использующих разные мобильные устройства - на базе IOS и Android.

def map_event(event):
    if event in ('main', 'call_made', 'request_call'):
        return event
    elif event in ('catalog/motobikes', 'catalog/tools'):
        return 'catalog'
    elif event in ('catalog/motobikes/CVO™ Limited® (FLHTKSE), 2020 Harley-Davidson', 'catalog/motobikes/Harley-Davidson, CVO™ Limited® (Flhtkse), 2020', 'catalog/motobikes/Road Glide Limited 114, Harley-Davidson (2020)', 'catalog/tools/49320-09 НАБОР ДУГ ЗАЩИТНЫХ ДВИГАТЕЛЯ, BLACK Harley Davidson', 'catalog/tools/55800646 РУЛЬ ДЛЯ МОТОЦИКЛА СОСТАВНОЙ', 'catalog/tools/67700455 KIT ФАРЫ МОТОЦИКЛА Harley Davidson'):
        return 'product_pages'
    else:
        return None
data['mapped_event'] = list(map(map_event, data['event']))
data['mapped_event'].isna().sum()
data.dropna(subset=['mapped_event'], inplace=True)
stages = ['main', 
          'catalog',
          'product_pages', 
          'call_made'
          'request_call']
retentioneering.config.update({
    'event_col':'mapped_event',
    'event_time_col':'timestamp',
    'user_col': 'client_id'
})
g1 = set(data[data['platform']=='android']['client_id'])
g2 = set(data[data['platform']=='iOS']['client_id'])

data.rete.funnel(targets = list(data['mapped_event'].unique()),
                 groups = (g1, g2),
                 group_names = ('androide', 'iOS'))

Что видим:

В результате получаем изображение, в котором наглядно представлены следующие данные:

• значительное число пользователей попадает на страницу каталога, минуя главную страницу
• пользователей, использующих устройства на базе Android значительно больше, чем пользователей, использующих устройства на базе IOS
• однако итоговая конверсия в целевое действие практически идентична для обоих типов устройств

Построим закрытую воронку

Для сравнения с предыдущим результатом в качестве критерия сегментации также используем тип мобильного устроства.

def df_for_closed_funnel(df, 
                         stages, 
                         event_col=retentioneering.config['event_col'], 
                         user_col=retentioneering.config['user_col'], 
                         gr_col=None):
    if gr_col:
        groups = list(df[gr_col].unique())
        for gr in groups:
            
            users_0 = set(df[(df[event_col]==stages[0])&(df[gr_col]==gr)][user_col])

            dict_user_stages = {}
            dict_user_stages[stages[0]] = users_0
            prev_users_stage = users_0

            for stage in stages[1:]:
                user_stage = set(df[(df[event_col]==stage)&(df[gr_col]==gr)][user_col])
                user_stage = user_stage - (user_stage - prev_users_stage)
                dict_user_stages[stage] = user_stage

                prev_users_stage = user_stage
                #удаляются юзеры, которых не было на предыдущем этапе
                df = df.drop(df[(df[event_col] == stage)&(df[gr_col]==gr)&(~df[user_col].isin(user_stage))].index)
                #если не указаны сегменты
    else:
        users_0 = set(df[df[event_col]==stages[0]][user_col])

        dict_user_stages = {}
        dict_user_stages[stages[0]] = users_0
        prev_users_stage = users_0

        for stage in stages[1:]:
            user_stage = set(df[df[event_col]==stage][user_col])
            user_stage = user_stage - (user_stage - prev_users_stage)
            dict_user_stages[stage] = user_stage

            prev_users_stage = user_stage
            #удаляются юзеры, которых не было на предыдущем этапе
            df = df.drop(df[(df[event_col] == stage)&(~df[user_col].isin(user_stage))].index)
        
    return df
data_closed = df_for_closed_funnel(
                        data, 
                        stages, 
                        gr_col='platform')
g1 = set(data_closed[data_closed['platform']=='android']['client_id'])
g2 = set(data_closed[data_closed['platform']=='iOS']['client_id'])

data_closed.rete.funnel(targets = list(data_closed['mapped_event'].unique()),
                 groups = (g1, g2),
                 group_names = ('android', 'iOS')
                )

Другие варианты использования

Меняя переменные в коде, вы можете строить воронки продаж по другим параметрам. Например, основываясь на данных в тестовом датасете, вы также можете посчитать конверсию по каналам привлечения или дислокации пользователя:

# например, закрытя воронка с использованием channel в качестве критерия
data_closed = df_for_closed_funnel(
                        data, 
                        stages, 
                        gr_col='channel')
# или закрытая воронка с использованием region в качестве критерия
data_closed = df_for_closed_funnel(
                        data, 
                        stages, 
                        gr_col='region')